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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicants submission filed on 
10/01/2007 has been entered. 

Response to Amendment 

2. This office action is in response to communications filed 10/01/2007. Claims 1 , 
35, 56, 63, 66, 67 and 70 are amended. Claims 2-9, 19, 20, 36-43, 48, 49 are original. 
Claim 14, 17, 18, 21, 46, 47, 50-52, 55, 56, 61, 62, 64, 65, 68 and 69 are previously 
presented. Claims 10-13, 15, 16, 22-34, 44, 45, 53, 54, 57-60 are cancelled. Claims 71 
and 72 are new. 

Response to Arguments 

3. Applicant's arguments filed 03/05/2007 have been fully considered but they are 
not persuasive. 
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Regarding Claims 1 and 35, the applicant argues that Wistendahl does not 
disclose the claimed object mapping table that stores an identifier to a corresponding 
one of a plurality of information data structures (Page 15). The applicant further argues 
that Wistendahl fails to teach or suggest the claimed plurality of information data 
structures and that even if N data were to identify the IDM program, it would not identify 
one of a plurality of IDM programs (Page 15). The applicant argues that the there is no 
suggestion or motivation to modify Wistendahl's N data were to identify the IDM 
program, it would not identify one of a plurality IDM programs. 

In response to the argument, Wistendahl discloses an object mapping table or N 
data information which includes N data indexed to specific type of interactive use of the 
media content (Column 7, lines 55-57, Column 6, lines 17-59, Figure 2). Microsoft 
Press 3 rd edition Computer Dictionary defines data structure as an organization 
scheme, such as a record or array, that can be applied to data to facilitate interpreting 
the data or performing operations on it. The N data is an organization scheme which 
applied to data to facilitate interpretation of the data or performing operations or for the 
interactive use of the media content. Wistendahl discloses the mask including an 
identifier or an object identifier for the hotspot with a A'(F) identifying object A or hotspot 
B'(F) identifying object B (Column 6, lines 17-38, Column 7, lines 1-10, Figure 2, Figure 
3). Wistendahl discloses a mask including an identifier to the object mapping table and 
the object mapping table having indicia from the corresponding mask to storing an 
identifier to a corresponding one of a pluriaty of information data structures including 
A'(Fj) or B'(Fj) (Column 6, lines 17-38, Column 10, lines 8-67) and the corresponding 
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information data structure including information associated with the particular video 
object (Column 10, lines 57-67, Column 11, lines 1-40, Column 12, lines 1-18, 40-49). 

Applicants argues that claims 63 and 67 are similar to limitations of claims 1 and 
35 and are now in condition for allowance for reasons discussed above with respect to 
claims 1 and 35. Applicant states that dependent claims are in condition for allowance 
because they depend on allowable base claims. 

In response to the arguments, please see response to arguments. 

Information Disclosure Statement 

4. The information disclosure statement filed 1-1 1-02, 5-13-04 fails to comply with 
37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; 
each non-patent literature publication or that portion which caused it to be listed; and all 
other information or that portion which caused it to be listed. It has been placed in the 
application file, but the information referred to therein has not been considered. 

The applicant has submitted many of the foreign patents and non-patent 
literature documents. However not all documents in the information disclosure 
statement filed 1-11-02, 5-13-04 have been submitted including IBM Technical 
Disclosure Bulletin's "User Controlled Display of Hypermedia Links Via Mouse Location" 
and "Visual Behavior of Multimedia Hypervideo Links for Motion Video". 
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5. The information disclosure statement (IDS) submitted 31 July 2006. The 
submission has met the minimum requirements of 37 CFR 1.97 and 37 CFR 1.98. 
Accordingly, the examiner is considering the IDS. 

It is noted, however, that the large number of references, represents a significant 
burden for the examination of the instant application. Therefore, the references have 
only been considered to the extent possible given limited examination resources. 
Should the applicant be aware that one or more of these documents is particularly 
relevant to patentability, it is requested that applicant's provide a concise explanation of 
why the English language information is being submitted and how it is understood to be 
relevant. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1-7, 17-21, 35-41, 50-52, 56 and 63-70 under 35 U.S.C. 103(a) as being 
unpatentable over Srinivasan et al (US 6,357,042' and hereafter referred to as 
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"Srinivasan") in view of Wistendahl et al (US 6,496,981 and hereafter referred to as 
"Wistendahl") 

Regarding Claims 1 and 35, Srinivasan discloses a hyperlinked broadcast 
system and a method of generating a hyperlinked video signal (Figure 1, Figure 14, 
Figure 18) comprising: a video headend or video source provides video information for a 
video program including a plurality of consecutive video frames (Figure 1,12, Column 7, 
lines 22-25, Column 8, lines 13-14, 38-49), 

an annotation system tracing or tracking movement of two or more video objects 
appearing in each of plurality of consecutive video frames from a first location to a 
second location (Column 8, lines 5-37) and generating annotation data (Column 6, lines 
8-19) and annotation data timing information (Column 7, lines 21-30, Column 8, lines 5- 
50), the annotation data including only one mask for each video frame of the plurality of 
consecutive frames (Column 5, lines 54-65, Figure 3, Figure 4), each mask being 
stamped with the corresponding annotation data timing information based on a frame 
time of the corresponding video frame (Abstract, Column 7, lines 21-30, Column 8, lines 
5-50), each mask including location (Figure 4, Column 8, lines 5-50) and graphic data of 
two or more graphic images to be overlaid on two or more video objects or entities in 
the corresponding video frame (Column 5, line 54-65, Column 6, lines 8-19, Column 7, 
lines 33-47); and 

an augmented video information transmission generator receiving the annotation 
data, the video information, and the annotation timing information, the augmented video 
information transmission generator generating an augmented video transmission signal 
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comprising annotation data, the annotation data timing information, and the video 
information, transmits the augmented video transmission signal to a receiver (Column 5, 
lines 54-65, Column 6, lines 29-33, Column 11, lines 36-37, Figure 18, Column 3, lines 
53-55, Column 4, lines 3-7, 17-24), 

wherein the augmented video information transmission generator associates the 
video information with the annotation data using the annotation data timing information 
(Column 5, lines 54-65, Column 6, lines 29-33, Column 11, lines 36-37), 

wherein the receiver receiving the augmented video transmission signal is 
program for each of the plurality of consecutive video frames (Column 36, lines 41-49, 
Column 21 , lines 34-52) to: 

compare a current annotation data timing information with a current frame time of 
a current video frame (Abstract, Column 4, lines 35-37, 55-62, Column 5, line 54-65, 
Column 6, lines 8-19, Column 7, lines 33-47, Column 12, lines 21-46, Column 36, lines 
41-49, Column 21, lines 34-52); 

retrieve the location and graphics data of the two or more graphics images from 
the identified mask if the comparison results in a match (Abstract, Column 4, lines 35- 
37, 55-62, Column 5, line 54-65, Column 6, lines 8-19, Column 7, lines 33-47, Column 
12, lines 21-46, Column 36, lines 41-49, Column 21, lines 34-52); and 

overlay the two or more graphics images on the corresponding two or more video 
object appearing in the video frame based on the retrieved location and graphics data, 
wherein the overlaying of the two or more graphics images is synchronized on a frame 
by frame basis with the movement of two or more video objects from the first location to 
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the second location over the plurality of consecutive video frames (Abstract, Column 4, 
lines 35-37, 55-62, Column 5, line 54-65, Column 6, lines 8-19, Column 7, lines 21-47, 
Column 12, lines 21-46, Column 36, lines 41-49, Column 21, lines 34-52). Srinivasan 
discloses annotation data comprises a plurality of object data packets (Column 21, lines 
14-25). 

Srinivasan is silent on the mask including an identifier to an object mapping table 
including one or more of the plurality of object data packets, the object mapping table 
including at least one entry with an indicia from the corresponding mask identifying a 
particular video object, the entry further storing an identifier to a corresponding one of a 
plurality information data structures included in one or more of the plurality object data 
packets, the information data structures including information for the particular video 
object. Wistendahl discloses on annotation data further includes a plurality of object 
data packets (Figure 3, 32, Column 11, lines 30-65, Column 17, lines 8-10), the mask 
further including an identifier or hotspot to an object mapping table or N data 
information (including values for several objects such as object A, A'(F), and object B, 
B'(F)) included in the object data packets (Column 6, lines 17-38, Column 7, lines 1-10, 
Figure 2, Figure 3, Column 11, lines 30-65, Column 17, lines 8-10). Microsoft Press 3 rd 
edition Computer Dictionary defines data structure as an organization scheme, such as 
a record or array, that can be applied to data to facilitate interpreting the data or 
performing operations on it. Wistendahl discloses the object mapping table including at 
least one entry with an indicia from the corresponding mask identifying a particular 
video object (Figure 2, Column 6, lines 17-38), the entry further storing an identifier to a 
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corresponding one of a plurality of information data structures included in one or more 
of the plurality of object data packets (Figure 2, Figure 3, Column 6, lines 17-38, 
Column 10, lines 8-67, Column 17, lines 1-13, Column 11, lines 30-65,), the information 
data structures including information for particular video object (Column 9, lines 59-67, 
Column 10, lines 1-67, Column 11, lines 1-40, Column 12, lines 1-18, 40-49). 

Therefore, it would have been obvious at the time the invention was made to one 
of ordinary skill in the art to modify Srinivasan to include the mask further including an 
identifier or hotspot to an object mapping table or N data information included in the 
object data packets (Column 6, lines 17-38, Column 7, lines 1-10, Figure 2, Figure 3, 
Column 11, lines 30-65, Column 17, lines 8-10), the object mapping table including at 
least one entry with an indicia from the corresponding mask identifying a particular 
video object (Figure 2,. Column 6, lines 17-38), the entry further storing an identifier to a 
corresponding one of a plurality of information data structures included in one or more 
of the plurality of object data packets (Figure 2, Figure 3, Column 6, lines 17-38, 
Column 10, lines 8-67, Column 17, lines 1-13, Column 11, lines 30-65,), the information 
data structures including information for particular video object (Column 9, lines 59-67, 
Column 10, lines 1-67, Column 11, lines 1-40, Column 12, lines 1-18, 40-49) as taught 
by Wistendahl in order to use media content for interactive television (Column 1 , lines 
12-14, 45-67) as disclosed by Wistendahl. 

Regarding Claims 63 and 67, Srinivasan discloses a hyperlinked broadcast 
system and a method of generating a hyperlinked video signal comprising: 
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a video source providing video information for a video program including a 
plurality of video frames (Figure 1, 12, Column 7, lines 22-25, Column 8, lines 13-14, 
38-49); 

receiving video information for a video program including a plurality of video 
frames (Figure 1, 12, Column 7, lines 22-25, Column 8, lines 13-14, 38-49) 

an annotation system generating annotation data and annotation data timing 
information (Column 8, lines 5-37), the annotation data including a plurality of masks 
and a plurality of object data packets (Figure 4, Figure 3, Column 8, lines 5-37), each 
mask corresponding to a particular video frame of a video program (Figure 3, 31 , 33, 
29, 27) and including graphics data associated with one or more video objects in the 
particular video frame (Column 5, line 54-65, Column 6, lines 8-19, Column 7, lines 33- 
47), each mask being associated with the corresponding annotation data timing 
information (Figure 4, Figure 3, Column 5, line 54-65, Column 6, lines 8-19, Column 7, 
lines 33-47), and an object mapping table included in at least a particular one of the 
plurality of object data packets (Figure 4), the object mapping table including an entry 
associated with each of the one or more video objects in the particular video frame 
(Figure 3, Figure 4), communicating the annotation data timing information, the 
annotation data and the video information to an augmented transmission generator 
(Figure 1, Figure 3, Figure 4) and 

an augmented video information transmission generator receiving the annotation 
data, the video information, and the annotation data timing information, the augmented 
video information transmission generator generating an augmented video transmission 
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signal comprising the annotation data, the annotation data timing information, and the 
video information, and transmitting the augmented video transmission signal to a 
receiver, wherein the augmented video information transmission generator associates 
the video information with the annotation data using the annotation data timing 
information (Column 5, lines 54-65, Column 6, lines 29-33, Column 11, lines 36-37, 
Figure 18, Column 3, lines 53-55, Column 4, lines 3-7, 17-24). Srinivasan discloses 
annotation data comprises a plurality of object data packets (Column 21, lines 14-25). 
Srinivasan is silent on object mapping table as claimed. 

Wistendahl discloses on annotation data further includes a plurality of object data 
packets (Figure 3, 32, Column 11, lines 30-65, Column 17, lines 8-10), the mask further 
including an identifier or a hyperlink to an object mapping table or N data information 
(including values for several objects such as object A, A'(F), and object B, B'(F))) 
(Column 6, lines 17-38, Column 7, lines 1-10). Microsoft Press 3 rd edition Computer 
Dictionary defines data structure as an organization scheme, such as a record or array, 
that can be applied to data to facilitate interpreting the data or performing operations on 
it. Wistendahl discloses the object mapping table including at least one entry with an 
indicia from the corresponding mask identifying a particular video object or one entry 
associated with each of the one or more video objects in the particular video frame 
(Figure 2, Column 6, lines 17-38), each entry in the object mapping table storing an 
identifier to one or one or more information data structures included in one or more of 
the plurality object data packets (Figure 2, Figure 3, Column 6, lines 17-38, Column 10, 
lines 8-67, Column 17, lines 1-13, Column 11, lines 30-65), the information data 
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structures including information for particular video object (Column 9, lines 59-67, 
Column 10, lines 1-67, Column 11, lines 1-40, Column 12, lines 1-18, 40-49). 

Therefore, it would have been obvious at the time the invention was made to one 
of ordinary skill in the art to modify Srinivasan to include the mask further including an 
identifier to an object mapping table or N data information (including values for several 
objects) (Column 6, lines 17-38, Column 7, lines 1-10) such as including at least one 
entry with an indicia from the corresponding mask identifying a particular video object 
(Figure 2, Column 6, lines 17-38), the entry storing an identifier to one or one or more 
information data structures included in one or more of the plurality object data packets 
(Figure 2, Figure 3, Column 6, lines 17-38, Column 10, lines 8-67, Column 17, lines 1- 

13, Column 11, lines 30-65), the information data structures including information for 
particular video object (Column 9, lines 59-67, Column 10, lines 1-56) as taught by 
Wistendahl in order to use media content for interactive television (Column 1 , lines 12- 

14, 45-67), to author exiting media content using tools to reduce development time 
(Column 2, lines 43-46) and so that media contact is kept uncorrupted so that the 
content can be displayed on any media delivery system or display platform (Column 3, 
lines 2-5) as disclosed by Wistendahl. 

Regarding Claims 2 and 36, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses the augmented video 
information transmission generator comprises a component, which inserts data into the 
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VBI or a vertical blanking insertion device (Column 35, lines 60-62, Column 36, lines 36- 
40). 

Regarding Claims 3 and 37, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses that the augmented 
video information transmission generator comprises a digital video data multiplexer 
(Column 36, lines 25-32). 

Regarding Claims 4 and 38, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses that he timing 
information comprises at least one of timestamp information and a frame number 
information (Column 3, Column 4, lines 3-16, Column 7, line 23, Column 8, lines 5-14). 

Regarding Claims 5 and 39, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses the programs 
comprise digital video signal (Column 36, lines 25-27). 

Regarding Claims 6 and 40, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses the programs 
comprise analog video signal (Column 36, lines 36-40). 

Regarding Claims 7, Srinivasan and Wistendahl disclose all the limitations of 
Claim 1. Srinivasan disclose a post production environment (Figure 1,11, Figure 18, 
251, 253, 255), and a headend comprising the augmented video information 
transmission generator (Column 6, lines 33-35, Figure 18, 259), the video information 
and annotation data timing information are combined by the post production 
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environment and transmitted to the headend (Column 7, lines 20-48, Column 8, lines 5- 
49, Column 35, lines 20-36). 

Regarding Claims 8, Srinivasan and Wistendahl disclose all the limitations of 
Claim 7. Wistendahl discloses that the headend is a cable headend (Column 16, lines 
6-8). 

Regarding Claim 14, Srinivasan and Wistendahl disclose all the limitations of 
Claims 1 and 35 respectively. Srinivasan discloses displaying annotation data in 
response to a viewer request (Column 6, lines 8-19, Column 12, lines 21-40). 

Regarding Claims 17 and 46, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses that the mask 
comprises location information of two or more objects or entities in an annotated video 
frame in the corresponding video frame (Column 5, lines 54-65, Column 7, lines 22-48, 
Column 8, lines 5-50). 

Regarding Claims 18 and 47, Srinivasan and Wistendahl disclose all the 
limitations of Claims 17 and 46 respectively. Srinivasan discloses the location 
information includes a graphics location reference that represents a fixed relation to a 
set of pixels associated with each object (Column 9, lines 19-55, Figure 4). 

Regarding Claims 19 and 48, Srinivasan and Wistendahl disclose all the 
limitations of Claims 18 and 47 respectively. Srinivasan discloses a table of pixels 
based on a tracking box and object (Figure 4). Therefore, it is necessarily included that 
the pixels include the upper left most pixel in the associated pixel set if the object that is 
being tracked is located at the upper most left corner (Figure 4, Figure 3). 
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Regarding Claims 20 and 49, Srinivasan and Wistendahl disclose all the 
limitations of claims 18 and 48 respectively. Srinivasan discloses a table of pixels 
based on a tracking box and object (Figure 4), if the object is in the center position then 
the centroid pixel is in the associated pixel set (Figure 4, Figure 3). Therefore, it is 
necessarily included that the pixels include the centroid pixel in the associated pixel set. 

Regarding Claim 41, Srinivasan and Wistendahl disclose all the limitations of 
Claim 35. Srinivasan discloses inserting the annotation data timing information in a 
vertical blanking interval of an analog video signal (Column 36, lines 36-41). 

Regarding Claim 43, Srinivasan and Wistendahl disclose all the limitations of 
Claim 35. Srinivasan discloses the transmitting the timing information and video 
information to a broadcast network and subsequently to the augmented video 
transmission generator (Column 5, lines 54-65, Column 6, lines 29-40, Column 7, lines 
22-25, Column 36, lines 25-40). 

Regarding Claims 21 and 50, Shoff discloses all the limitations of Claims 1 and 
35 respectively. Srinivasan discloses that the mask comprises location information 
about an object in a video frame to be annotated or supplemental content to be added 
to video (Figure 4). Srinivasan is silent on the shape information. In analogous art, 
Wistendahl discloses that location and shape information of an object in the video frame 
(Column 10, lines 8-35). Therefore, it would have been obvious at the time the 
invention was made to one of ordinary skill in the art to modify Shoff to include shape 
information of an object in the video frame (Column 10, lines 8-35) as taught by 
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Wistendahl in order to use media content for interactive television (Column 1 , lines 12- 
14, 45-67) as disclosed by Wistendahl. 

Regarding Claim 51, Srinivasan and Wistendahl disclose all the limitations of 
Claim 50. Wistendahl discloses the shape information is represented by a hyper link 
(Column 10, lines 36-56). The hyperlink can be a graphical overlay of the object 
(Column 9, lines 28-33). 

Regarding Claim 52, Srinivasan and Wistendahl disclose all the limitations of 
Claim 50. Wistendahl discloses the shape information is represented by an outline of 
the object (Column 10, lines 25-28). 

Regarding Claim 56, Srinivasan and Wistendahl disclose all the limitations of 
Claim 1. Wistendahl discloses the receiver being configured to overlay a graphics 
image on a particular video frame for the particular video object based on the graphics 
data included in the corresponding mask (Column 9, lines 59-67, Column 10, lines 1- 
56), retrieve the identifier of the object mapping table from the corresponding mask 
response to a user selection associated with the overlaid graphics image (Column 6, 
lines 60-67, Column 7, lines 1-10, Column 9, lines 59-67, Column 10, lines 1-56); 
retrieve the object mapping table based on the retrieved identifier (Figure 3, Figure 7a); 
identify the indicia in the corresponding mask for the particular video object for which 
the graphics image was overlaid (Column 15, lines 3-5); locate the entry in the object 
mapping table with the identified indicia (Figure 5b, 51b, Figure 7a, Figure 7B); ); 
retrieving from the located entry the identify of the corresponding information data 
structures identified by the retrieved identifier (Column 6, lines 60-67, Column 7, lines 1- 
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10, Column 15, lines 3-5); locate the entry in the object mapping table with the identified 
indicia (Figure 5b, 51b, Figure 7a, Figure 7B); identify the information data structures 
associated with the located entry (Figure 5a, Figure 5b); retrieve the information data 
structures identified by retrieved identifier (Figure 2, Figure 3); and display information 
based on the retrieved information data structure on the display device (Figure 7a, 
Column 13, lines 54-62). 

Regarding Claims 64 and 68, Srinivasan and Wistendahl disclose all the 
limitations of Claims 63 and 67 respectively. Srinivasan discloses comparing a current 
annotation data timing information with a current frame time of a current video frame 
(Abstract, Column 4, lines 35-37, 55-62, Column 5, line 54-65, Column 6, lines 8-19, 
Column 7, lines 33-47, Column 12, lines 21-46, Column 36, lines 41-49, Column 21, 
lines 34-52); retrieve the location and graphics data of the graphics images from the 
identified mask if the comparison results in a match (Abstract, Column 4, lines 35-37, 
55-62, Column 5, line 54-65, Column 6, lines 8-19, Column 7, lines 33-47, Column 12, 
lines 21-46, Column 36, lines 41-49, Column 21, lines 34-52); and overlay one or more 
graphics images generated based on the retrieved graphics data on the one or more 
video objects appearing in the current video frame (Abstract, Column 4, lines 35-37, 
55-62, Column 5, line 54-65, Column 6, lines 8-19, Column 7, lines 21-47, Column 12, 
lines 21-46, Column 36, lines 41-49, Column 21, lines 34-52). 

Regarding Claims 65 and 69, Srinivasan and Wistendahl disclose all the 
limitations of Claims 64 and 68 respectively. Srinivasan discloses the receiver is further 
programmed to draw the one or more graphics images on a frame-by-frame basis 
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based on the graphics data included in a plurality of masks, the drawing of the one or 
more graphics images being synchronized to the corresponding video frame based on 
the annotation data timing information associated with the plurality of masks (Abstract, 
Column 4, lines 35-37, 55-62, Column 5, line 54-65, Column 6, lines 8-19, Column 7, 
lines 21-47, Column 12, lines 21-46, Column 36, lines 41-49, Column 21, lines 34-52). 
Wistendahl discloses drawing one or more graphic images on a frame-by-frame basis 
based on the graphics data included in a plurality of masks (Column 14, lines 10-21 , 
Column 10, lines 8-61). 

Regarding Claims 66 and 70, Srinivasan and Wistendahl disclose all the 
limitations of Claims 65 and 69 respectively. Wistendahl discloses the receiver being 
configured to overlay a graphics image on a particular video frame for the particular 
video object based on the graphics data included in the corresponding mask (Column 9, 
lines 59-67, Column 10, lines 1-56), receive a user selection associated with one of the 
overlaid graphics images for a particular video frame (Column 15, lines 3-5), retrieve the 
identifier of the object mapping table from the corresponding mask response to a user 
selection associated with the overlaid graphics image (Column 6, lines 60-67, Column 
7, lines 1-10, Column 9, lines 59-67, Column 10, lines 1-56); retrieve the object mapping 
table based on the retrieved identifier (Figure 3, Figure 7a); locate the entry in the object 
mapping table for the video object associated with the user selection (Column 6, lines 
60-67, Column 7, lines 1-10, Figure 5b, 51b, Figure 7a, Figure 7B, Column 13, lines 1- 
25, Column 14, lines 10-21); retrieving from the located entry the identify of the 
corresponding information data structures identified by the retrieved identifier (Column 
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6, lines 60-67, Column 7, lines 1-10, Column 15, lines 3-5); locate the entry in the object 
mapping table with the identified indicia (Figure 5b, 51b, Figure 7a, Figure 7B); identify 
the information data structures associated with the located entry (Figure 5a, Figure 5b); 
retrieve the information data structures identified by retrieved identifier (Figure 2, Figure 
3); and display information based on the retrieved information data structure on the 
display device (Figure 7a, Column 13, lines 54-62). 

8. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Srinivasan in view of Wistendahl, as applied to Claim 7 above, further in view of Shoff et 
al (US 6,240,555 and hereafter referred to as "Shoff). 

Regarding Claims 9, Srinivasan and Wistendahl disclose all the limitations of 
Claim 7. Srinivasan and Wistendahl do not to disclose the headend is a satellite 
headend. In analogous art, Shoff discloses a post production environment (Figure 2, 
22), a broadcast network (Column 4, lines 43-50), and the post production environment 
combines video data and synchronized timing data and transmits to a headend or node 
(Column 4, lines 43-50). Shoff discloses that the node is a satellite headend (Column 4, 
lines 45-52). Therefore, it would have been obvious at the time the invention was made 
to one of ordinary skill in the art to modify Srinivasan to include that the headend is a 
satellite headend (Column 4, lines 45-52) as taught by Shoff in order to enable viewer 
interactively with video program (Column 1, lines 8-14) as disclosed by Shoff. 
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9. Claim 42 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Srinivasan in view of Wistendahl, as applied to Claim 35 above, further in view of Oguro 
et al (US 2001/0033739 and hereafter referred to as "Oguro"). 

Regarding Claim 42, Srinivasan and Wistendahl disclose all the limitations of 
Claim 35. Srinivasan disclose inserting timing information into the VBI of an analog 
signal (Column 36, lines (Column 25-40). Srinivasan and Wistendahl are silent the 
insertion of data in the vertical ancillary data or VBI of a digital video signal. Oguro 
discloses a television broadcast system, which transmits digital video signals to the user 
(Page 5, paragraph 0077) and inserts data into the VBI of a digital video signal (Page 5, 
paragraph 0076). Therefore, it would have been obvious at the time the invention was 
made to one of ordinary skill in the art to modify the combination to include inserting 
information in the VBI of a digital video signal (Page 5, paragraph 0076) as taught by 
Oguro in order to provide copy protect television broadcast programs of a digital video 
signal (Page 1 , paragraphs 0001-0006) as disclosed by Oguro. 

10. Claims 61, 62 and 72 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Srinivasan in view of Wistendahl, as applied to Claims 1 and 35 above, further in 
view of Kaiser et al (US 6,615,408 and hereafter referred to as "Kaiser"). 

Regarding Claims 61 and 62, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses overlaying two or more 
graphics images for alerting a viewer of the interactive data (Column 6, lines 6-19). 
Srinivasan and Wistendahl do not explicitly disclose the overlaying of the graphics 
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images is for alerting a viewer of the interactive data associated prior to the viewer 
transmitting an interactive command with respect to one of video object or video 
objects. Kaiser discloses overlaying of the two or more graphics images is for alerting a 
viewer of the interactive data associated with the two or more video objects prior to the 
viewer transmitting an interactive command with respect to one of the two or more video 
objects (Figure 6B, Column 10, lines 20-41). Therefore, it would have been obvious at 
the time the invention was made to one of ordinary skill in the art to modify the 
combination to include overlaying of the two or more graphics images is for alerting a 
viewer of the interactive data associated with the two or more video objects prior to the 
viewer transmitting an interactive command with respect to one of the two or more video 
objects (Figure 6B, Column 10, lines 20-41) as taught by Kaiser in order display a 
locator or locators for a video object or objects without clutter video (Column 1 , lines 8- 
1 1 f 34-40) as disclosed by Kaiser and to make it easier for a user so that they do not 
have to determine what is interactive and provides more information. 

Regarding Claim 72, Srinivasan and Wistendahl disclose all the limitations of 
Claim 1 . Srinivasan discloses overlaying two or more graphics images for alerting a 
viewer of the interactive data in which a mask includes data enabled to identify video 
objects in a particular video shot (Column 6, lines 6-19). Srinivasan and Wistendahl do 
not disclose a visibility bit indicative of visually identify an object. Kaiser discloses a 
mask or hotspot includes a visibility bit or highlight information indicative of whether 
video objects appearing in the corresponding video frame are enabled for being visually 
identified for a particular video shot (Column 10, lines 20-42). Therefore, it would have 
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been obvious at the time the invention was made to one of ordinary skill in the art to 
modify the combination to include a visibility bit indicative of whether video objects 
appearing in the corresponding video frame are enabled for being visually identified for 
a particular video shot (Column 10, lines 20-42) as taught by Kaiser in order display a 
locator or locators for a video object or objects without clutter video (Column 1 , lines 8- 
1 1 , 34-40) as disclosed by Kaiser and to make it easier for a user so that they do not 
have to determine what is interactive and provides more information. 

11. Claim 71 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Srinivasan in view of Wistendahl, as applied to Claim 1 above, further in view of Reimer 
et al (US 5,553,221 and hereafter referred to as "Reimer"). 

Regarding Claim 71, Srinivasan and Wistendahl disclose all the limitations of 
Claim 1. Wistendahl disclose a corresponding data structure (Figure 2, Figure 3). 
Srinivasan and Wistendahl are silent on the corresponding information data structure is 
an object properties table storing one or more second entries, wherein each of the one 
or more second entries includes an information category and an identifier to a second 
one of the plurality of information data structures providing details for the information 
category. Microsoft Press 3 rd edition Computer Dictionary defines data structure as an 
organization scheme, such as a record or array, that can be applied to data to facilitate 
interpreting the data or performing operations on it. In analogous art, Reimer discloses 
on the corresponding information data structure is an object properties table (Figure 8, 
802, Figures 10A-10C, Figure 11, 1106) storing one or more second entries (Figures 
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10A-10C, Column 18, lines 45-57), wherein each of the one or more second entries 
includes an information category and an identifier to a second one of the plurality of 
information data structures providing details for the information category (Column 18, 
lines 45-56, Figure 10A, 1002, Figure 10B, 1012, Figure 10C, 1018, Figure 8, 802, 
Column 17, lines 40-47). Therefore, it would have been obvious at the time the 
invention was made to one of ordinary skill in the art to modify the combination to 
include corresponding information data structure is an object properties table (Figure 8, 
802, Figures 10A-10C, Figure 11, 1106) storing one or more second entries (Figures 
10A-10C, Column 18, lines 45-57), wherein each of the one or more second entries 
includes an information category and an identifier to a second one of the plurality of 
information data structures providing details for the information category (Column 18, 
lines 45-56, Figure 10A, 1002, Figure 10B, 1012, Figure 10C, 1018, Figure 8, 802, 
Column 17, lines 40-47) as taught by Reimer in order to link and present information of 
movies with underlying source information (Column 1 , lines 8-1 1 , 34-40) as disclosed by 
Reimer. 



Double Patenting 



12. The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. A nonstatutory 
obviousness-type double patenting rejection is appropriate where the conflicting claims 
are not identical, but at least one examined application claim is not patentably distinct 
from the reference claim(s) because the examined application claim is either anticipated 
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by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 
F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 
USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 
1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 
F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) 
may be used to overcome an actual or provisional rejection based on a nonstatutory 
double patenting ground provided the conflicting application or patent either is shown to 
be commonly owned with this application, or claims an invention made as a result of 
activities undertaken within the scope of a joint research agreement. 

Effective January 1 , 1994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 

13. Claim 1 is rejected on the ground of nonstatutory obviousness-type double 
patenting as being unpatentable over claim 1 of U.S. Patent No. 7,249,367 in view of 
Srinivasan. 

The instant application's limitations of " a hyperlinked broadcast system 
comprising: a video source providing video program including a plurality of consecutive 
video frames" is met by the limitations of "A hyperlinked video broadcast system" of U.S. 
Patent No. 7,249,367. 

The instant application's limitations of "an annotation system generating 
annotation data, the annotation data including only one mask for each video frame of 
the plurality of consecutive frames, each mask being stamped with the corresponding 
annotation data timing information based on a frame time of the corresponding video 
frame, each mask including location and graphic data of two or more graphic images to 
be overlaid on two or more video objects or entities in the corresponding video frame; 
the annotation data further including a plurality of object data packets, mask including 
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an identifier to an object mapping table including one or more of the plurality of object 
data packets, the object mapping table including at least one entry with an indicia from 
the corresponding mask identifying a particular video object, the entry further storing an 
identifier to a corresponding one of a plurality information data structures included in 
one or more of the plurality object data packets, the information data structures 
including information for the particular video object " is met by the limitations " 
a mask generator generating a plurality of masks, each mask corresponding to a 
particular video frame of a video program, each mask including graphics data 
associated with one or more video objects in the particular video frame, each mask 
including an identifier to an object mapping table including an entry associated with 
each of the one or more video objects in the particular video frame, each entry in the 
object mapping table referencing one or more information data structures including 
information associated with the corresponding video object; an annotation source 
providing a plurality of object data packets including the object mapping table and the 
one or more information data structures for the one or more video objects" of U.S. 
Patent No. 7,249,367. 

The instant application's limitations of an augmented video information 
transmission generator receiving the annotation data, the video information, the 
augmented video information transmission generator generating an augmented video 
transmission signal comprising annotation data, the and the video information, transmits 
the augmented video transmission signal to a receiver" is met by limitation "a transmitter 
transmitting the television broadcast signal" of U.S. Patent No. 7,249,367. 
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wherein the augmented video information transmission generator associates the 
video information with the annotation data using the annotation data timing information, 
wherein the receiver receiving the augmented video transmission signal is program for 
each of the plurality of consecutive video frames to:; retrieve the location and graphics 
data of the two or more graphics images from the identified mask if the comparison 
results in a match; and overlay the two or more graphics images on the corresponding 
two or more video object appearing in the video frame based on the retrieved location 
and graphics data, wherein the overlaying of the two or more graphics images is 
synchronized on a frame by frame basis with the movement of two or more video 
objects from the first location to the second location over the plurality of consecutive 
video frames" is met by limitations "wherein the receiver is programmed to retrieve and 
overlay the graphics image corresponding to the particular video object on the 
corresponding video frame, receive viewer actuation of the overlaid graphics image, and 
in response to the viewer actuation, review the indicia in the particular information data 
structure corresponding to the particular video object for determining whether the 
particular video object is linked to one of the plurality of multiplexed program streams, 
and in response to a determination that the particular video object is linked to one of the 
plurality of multiplexed program streams, retrieve from the data particular information 
data structure the identifier of the particular one of the plurality of multiplexed program 
streams, and switch from presenting a current multiplexed program stream to 
presenting the particular one of the plurality of multiplexed program streams" of U.S. 
Patent No. 7,249,367. 
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The instant application is missing including "a plurality of multiplexed program 
streams, an identifier for a particular one of the plurality of multiplexed program streams, 
a particular one of the information data structures including an indicia indicative that a 
particular one of the one or more video objects is linked to one of the plurality of 
multiplexed program streams, and an encoder encoding the plurality of masks and the 
plurality of object data packets into a television broadcast signal; a transmitter 
transmitting the plurality of multiplexed program streams to a receiver." It would be 
obvious to modify U.S. Patent No. 7,249,367 to make the claim broader. 

The instant application's "an annotation system tracking movement of two or 
more video objects appearing in each of plurality of consecutive video frames from a 
first location to a second location and generating annotation data timing information, an 
augmented video information transmission generator generating the annotation timing 
information, the augmented video information transmission generator generating an 
augmented video transmission signal comprising annotation data, the annotation data 
timing information, and the video information, transmits the augmented video 
transmission signal to a receiver, wherein the receiver compare a current annotation 
data timing information with a current frame time of a current video frame" are additional 
features. 

It would be obvious to include the limitations in U.S. Patent 7,249,367 as they are 
disclosed by prior art. Srinivasan discloses an annotation system generating annotation 
data timing information (Column 7, lines 21-30, Column 8, lines 5-50), the augmented 
video information transmission generator receiving the annotation timing information, 
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the augmented video information transmission generator generating an augmented 
video transmission signal comprising annotation data, the annotation data timing 
information, and the video information(Column 5, lines 54-65, Column 6, lines 29-33, 
Column 11, lines 36-37, Figure 18, Column 3, lines 53-55, Column 4, lines 3-7, 17- 
24),wherein the augmented video information transmission generator associates the 
video information with the annotation data using the annotation data timing information 
(Column 5, lines 54-65, Column 6, lines 29-33, Column 11, lines 36-37),wherein the 
receiver compare a current annotation data timing information with a current frame time 
of a current video frame (Abstract, Column 4, lines 35-37, 55-62, Column 5, line 54-65, 
Column 6, lines 8-19, Column 7, lines 33-47, Column 12, lines 21-46, Column 36, lines 
41-49, Column 21, lines 34-52). 

Conclusion 

14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Farzana E. Hossain whose telephone number is 571- 

272- 5943. The examiner can normally be reached on Monday to Friday 7:00 am to 
3:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Christopher Kelley can be reached on 571-272-7331. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 

273- 8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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